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Abstract 

Professional development (PD), which includes coaching, has demonstrated the capacity to affect preschool teachers’ use of 
evidence-based practices. The present study explored how coaches facilitated conversations within practice-based coaching 
(PBC) partnerships. A direct behavioral observation coding system was developed to investigate (a) the proportion of time 
spent in different conversational foci, including who initiated the conversation; (b) coach verbal behaviors; and (c) whether 
the conversation foci, initiations, and coach verbal behavior changed across three occasions for seven coach—teacher 
dyads. Results from the present study indicate coaches spent the largest proportion of time engaging in conversations 
with teachers focused on reflection and feedback, followed by goal setting and action planning. The coaches used verbal 
behaviors (supportive and constructive feedback, clarifying questions) as required by the coaching protocol across all 
sampled occasions. Variation in conversation foci, initiations, and coach verbal behavior across three sampled occasions 


was evident. 
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The provision of quality early learning experiences is con- 
tingent on teachers who are well prepared at entry and 
supported by ongoing professional development (PD) to 
implement evidence-based practices as intended (Winton 
et al., 2015). Intentional alignment between the desired 
outcomes of PD and the type and intensity of PD pro- 
vided is necessary, so resources are used efficiently and 
effectively to enhance teachers’ practice implementation 
(Winton et al., 2015). Workshops, in combination with 
follow-up implementation support using strategies such as 
coaching, are increasingly prevalent in early childhood 
and early childhood special education research, where 
enhanced knowledge and application of practices in the 
classroom are the desired outcomes (Artman-Meeker 
et al., 2015; Snyder et al., 2012). Snyder and colleagues 
(2012) conducted a review of early childhood PD studies 
and found that 74% of the 256 studies reviewed included 
some type of initial training (e.g., workshop) plus follow- 
up support, with the largest proportion of studies (51.6%) 
reporting the use of coaching as a form of follow-up sup- 
port. Similarly, Artman-Meeker and colleagues (2015) 
conducted a review of the early childhood coaching litera- 
ture and found that of the 49 studies reviewed, 89.9% 
reported coaching as follow-up to an initial training. The 
authors of these literature reviews and other researchers in 


the field (e.g., Zaslow et al., 2010) contend that enhanced 
reporting of PD’s active ingredients, including coaching, 
and the extent to which the specified PD components are 
implemented with fidelity are needed to advance PD 
science. 

The absence of detailed reporting about the active ingre- 
dients of PD and coaching, in particular, presents two sig- 
nificant challenges: (a) replication and meta-analytic studies 
cannot be conducted to accumulate knowledge about which 
ingredients are important for whom and under what condi- 
tions and (b) policy makers and program leaders might be 
misled about the fiscal and human resources required to 
achieve outcomes in applied contexts similar to those found 
in rigorous research studies, preventing the successful 
installation and scaling up of promising PD interventions. 
Enhanced documentation and reporting of the components 
of PD interventions and of coaching specifically should 
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address three dimensions: content, structure, and process 
(Powell & Diamond, 2013; Snyder et al., 2011). 

The content dimension represents the knowledge or 
teaching practices that are the PD foci. This dimension 
often includes information about developmentally appro- 
priate child behaviors and teaching practices that support 
what children need to know or be able to do. The content 
dimension has been reported by researchers through the use 
of PD scripts, fidelity checklists, and printed manuals or 
guides for coaches and teachers (e.g., Powell & Diamond, 
2013; Snyder et al., 2015). 

The structural dimension characterizes the number, dura- 
tion, frequency, and delivery format of PD, which make up 
the dose and intensity of implementation support. The 
structural dimension might also include a specified coach- 
ing framework, with defined components or activities 
designed to support teachers’ acquisition of knowledge or 
use of teaching practices. The structural dimension can be 
operationalized and documented through the use of proto- 
cols, fidelity checklists, and self-report logs when explicit 
instruction is provided on how to record the frequency, 
duration, and contact with teachers (Durlak & DuPre, 2008; 
Snyder et al., 2015). 

The process dimension is focused on how those who 
deliver PD, including coaches, use protocols and other 
implementation supports to facilitate interactions that pro- 
mote teacher knowledge and use of teaching practices. The 
content and structural dimensions guide the composition of 
actions, behaviors, and materials used by the PD facilitator 
or coach. The process dimension maintains a degree of flex- 
ibility to respond to the knowledge, preferences, motiva- 
tions, and learning goals of individual practitioners (Powell 
& Diamond, 2013; Snyder et al., 2015). Responsivity to 
each participant’s learning history is aligned with the sci- 
ence of how people learn (National Academies of Sciences, 
Engineering, and Medicine [NASEM], 2018), but presents 
a challenge for accurate and feasible documentation. 
Fidelity checklists can be used to record adherence but 
often provide limited options for recording acceptable 
forms of variability. 

A review of contemporary literature demonstrates direct 
behavioral observation coding systems hold promise for 
characterizing the process dimension of coaching, particu- 
larly the interactions that occur between the coach and 
teacher. Observational coding systems in early childhood 
PD primarily focus on two outcomes: (a) defining and 
describing the verbal behaviors of coaches as they interact 
with families and teachers during home-based or classroom 
coaching sessions and (b) examining the accuracy of 
coaches’ self-report data. Researchers across several studies 
(e.g., Friedman et al., 2012; Oborn & Johnson, 2015) used 
a direct behavioral observation coding system aligned with 
the Family-Guided Routines-Based Intervention (FGRBI; 
Woods, 2005). The authors used procedures developed by 


Salisbury et al. (2008) to adapt the FGRBI checklist for 
video coding focused on nine coaching strategies. In these 
studies, the entire early intervention (EI) session is video- 
recorded and coded using a 30-s interval coding procedure. 
Participants were providers situated within a triadic coach- 
ing partnership, coaching families on their existing casel- 
oad around practices to support the child’s Individualized 
Family Service Plan (IFSP). 

Similar to Woods and colleagues, Campbell and Coletti 
(2013) examined EI providers’ self-reported versus 
observed use of five coaching or “caregiver-teaching” strat- 
egies. Providers submitted self-selected video segments 
(i.e., 1-40 min) of EI sessions with families on their exist- 
ing caseload. Providers and independent observers identi- 
fied coaching strategies represented in each video segment. 
Inter-rater agreement was calculated to determine the extent 
to which providers could implement and accurately self- 
report their use of coaching strategies. Jayaraman et al. 
(2015) conducted an exploratory study of coach—teacher 
dyads’ verbal and gestural behavior. Dyads were situated 
in preschool, child care, and home-based settings. This 
study differed from Woods and colleagues’ research in that 
the sample included coach—teacher dyads in addition to 
triadic provider—family—child partnerships and coaches 
supported coachees around Individualized Education 
Programs (IEPs), IFSPs, classroom environments, and 
social-emotional development. Each coach submitted one 
video of a self-selected debrief meeting with an existing 
coachee on their caseload. Videos were coded using the 
Early Childhood Coaching Conversations (ECCC; Knoche 
& Bainter, 2012) interval coding system to examine the 
coach’s implementation of 13 behaviors. 

Overall, these studies have demonstrated the utility of 
direct behavioral observation coding systems for character- 
izing coaches’ process behaviors within a coaching partner- 
ship. Specifically, these studies used partial-interval coding 
systems to look at coaching practices at a single occasion or 
in aggregate across occasions. The majority of available stud- 
ies focused on triadic home-based settings to coach families 
around IFSP goals. Jayaraman et al. (2015) included some 
school-based teachers in their sample, but an identified set of 
teaching practices around which coaching occurred was not 
provided. None of the identified studies explored duration 
codes or practice-based coaching (PBC; Snyder et al., 2015). 

In the present study, we sought to expand the available 
literature by exploring how seven coaches facilitated con- 
versations with school-based preschool teachers about an 
identified set of evidence-based embedded instruction teach- 
ing practices within a 15-week PBC partnership. Specifically, 
we used a continuous timed-event video-based, direct 
behavioral observation coding system to investigate the 
(a) proportion of time spent in conversational foci, includ- 
ing who initiated the conversation; (b) type and rate of 
coach verbal behavior; and (c) whether conversation foci, 
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initiations, and coach verbal behavior varied across three 
occasions (i.e., early, middle, and late coaching sessions). 
The use of video-based direct behavioral observation coding 
systems to characterize the interaction behaviors of coaches 
and coachees has the potential to advance research focused 
on coaching processes by quantifying how coaches promote 
providers’ or teachers’ acquisition and use of practices in 
applied contexts, information that, in turn, could have 
instructional utility for coach PD. 


Method 
Study Context 


Data used to conduct the present study were collected 
within the first year of a randomized controlled trial (RCT) 
focused on examining the effects of a PD intervention on 
preschool teachers’ use of embedded instruction teaching 
practices with preschool children with disabilities. In Year 
1, 44 preschool teachers from two southeastern states, 
located in proximity to two universities, were randomly 
assigned at each site to one of the three conditions: (a) 16 
hr of workshops, implementation guides, a web-based mul- 
timedia tool kit, plus | week of onsite PBC (n = 15); (b) 16 
hr of workshops, implementation guides and materials, a 
web-based multimedia tool kit, plus | week of web-medi- 
ated self-coaching (n = 14); or (c) district-provided PD (n 
= 15). The onsite PBC condition during Year | is the focus 
of the present study. 


Participants 


In the RCT, 15 teachers were randomly assigned to the 
onsite PBC condition in the first year of the project. Each of 
the seven coaches worked with one to three teachers at their 
university-based site. For the present study, interactions dur- 
ing the debrief meeting between each coach and one teacher 
were coded. For each coach, the teacher who received the 
dose of coaching most aligned with the PBC protocol used 
in the RCT (i.e., one onsite coaching session per week for 15 
consecutive school weeks) was selected for analyses in the 
present study. Teachers (NV = 7) were credentialed public pre- 
school teachers in four districts in two southeastern states. 
All teachers were White females between the ages of 23 to 
54 years. All teachers held degrees in education; four teach- 
ers held a bachelor’s degree and three teachers held a mas- 
ter’s degree. All teachers had children with and without 
disabilities in their classroom. At the beginning of the study, 
teachers’ experience working with children from the ages of 
birth to 5 years ranged from | month to 7 years. 

Coaches in the present study (NV = 7) were affiliated with 
one of two universities. All coaches were White females 
between the ages of 28 to 59 years. Six coaches had between 
9 months to 30 years of experience as a classroom teacher 


or therapist working with children from the ages of birth to 
5 years. One coach had second grade teaching experience. 
Six coaches held a master’s degree or higher, five coaches 
held a teaching credential, and five coaches had prior coach- 
ing or consultative experiences. Fidelity of coaching imple- 
mentation checklists were completed by trained project 
staff using video-recorded debrief meetings for a randomly 
selected 33% of all coaching sessions in the RCT. Coaches’ 
fidelity of implementation of the PBC protocol was high. In 
the RCT, the mean percentage of PBC fidelity checklist indi- 
cators implemented in Year | was 91.4% (SD = 5.8) across 
all selected sessions and for dyads in the present study was 
91.6% (SD = 5.2). 


Measures 


Coaching Practices Observation Tool—Research Version | 
(CPOT-RVI). The CPOT-RVI is a continuous, timed-event, 
observational coding system designed to quantify the dura- 
tion of coaching conversations, the proportion of coach and 
teacher initiations, and the rate of coach verbal behaviors 
(Shannon & Snyder, 2016). The operational definitions for 
the CPOT-RV1 codes are aligned with the essential compo- 
nents of the PBC framework. There are seven mutually 
exclusive and exhaustive duration codes (see Table 1). Five 
duration codes represent bidirectional conversation between 
the coach and the teacher, one code (i.e., Summarizing) rep- 
resents consecutive statements made by the coach only, and 
Uncodeable indicates the video could not be coded (e.g., 
talk cannot be heard). The onset of a duration code occurs 
when the coach or teacher has three verbal turns with new 
focus. Coders then map backwards to indicate the onset. 
The offset of the duration code is triggered by the onset of a 
new code. /nitiation is a binary code, which is used follow- 
ing the onset of a new duration code to signify whether the 
coach or teacher initiated the change in conversation focus. 
There are eight event codes, which provide information 
about the coach’s verbal behavior (see Table 2). Event 
codes are counted at the onset of the first occurrence of a 
unique coach verbal behavior. 


Demographic questionnaire. Teachers and coaches com- 
pleted demographic questionnaires, and coaches submitted 
information about each coaching session (i.e., video of 
debrief meeting, coaching log, observation notes, action 
plan, follow-up email) as part of the RCT. The demographic 
and coaching session data were used to support teacher 
selection and the interpretation of the aggregate CPOT-RVI 
data in the present study. 


Procedures 


Coach training and support. Prior to supporting teachers in 
the RCT, coaches attended 16 hr of training over 2 days 
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Table 1. Coaching Practices Observation Tool-Research Version |.0 Operational Definitions—Conversation Focus (Duration 


Codes). 
Code Operational definition 
Personal/ A verbal exchange initiated by the coach or teacher to engage in pleasantries or share personal 


Pleasantries 
Coaching Process 


Goal Setting and 
Action Planning 


Problem-Solving 


Reflection and 
Feedback 


Summarizing 


Uncodeable 


information that is not related to the implementation of targeted teaching practices. 

A verbal exchange initiated by the coach or teacher where participants discuss the coaching process 
including individual roles, activities to be completed, scheduling, and materials. 

A verbal exchange initiated by the coach or teacher in which (a) goals or action steps are identified and 
revised AND/OR (b) the coach provides goal focused instruction designed to increase the teachers’ 
capacity to implement targeted teaching practices. 

A verbal exchange initiated by the coach or teacher to discuss a problem related to implementation of 
targeted teaching practices or children’s individual needs are identified AND potential solutions to the 
identified problem are generated by the participants. 

A verbal interaction initiated by the coach or teacher in which (a) participants engage in a discussion 
about what occurred during the observation AND (b) share their personal opinions, perspectives, or 
feelings related to the classroom or targeted teaching practices. 

The coach makes three or more statements about what occurred in the observation OR summarizes what 
has occurred previously during the debriefing conversation. This code does not include any teacher talk. 

The coach or teacher behavior cannot be accurately recorded because one of the following events 
occurred for 5 or more seconds: (a) Out of frame—the coach or the teacher leaves the frame of the 
camera; (b) Talk is directed to someone other than the participating coach or teacher. 


Table 2. Coaching Practices Observation Tool-Research Version |.0 Operational Definitions—Coach Verbal Behaviors (Event 


Codes). 


Code 


Operational definition 


Neutral Statements 
General Praise 


Supportive Feedback 
Constructive 
Feedback 
Instructional 
Statements 
Clarifying Questions 


Probing Questioning 


Demonstration 


The coach makes neutral verbal statements of two or more words. 

The coach provides general positive statements or statements of agreement about something the 
teacher said or did during the observation or a product (e.g., planning form) the teacher made 
that is aligned with the implementation of targeted practices or classroom quality. 

The coach provides specific praise by describing positive aspects of something the teacher said or 
did during the observation, or a product (e.g., planning form) the teacher made that is aligned 
with the implementation of targeted teaching practices or classroom quality. 

The coach responds to something the teacher said or did during the observation or a product (e.g., 
planning form) the teacher made that is either not aligned with targeted teaching practices or not 
implemented as intended by providing guidance on how to enhance implementation fidelity. 

The coach makes statements to inform or teach about how to enhance future implementation of 
the targeted teaching practices or classroom quality. 

The coach asks the teacher a question to confirm understanding, actions, or scheduling. 

The coach asks the teacher a question to encourage him/her to share personal opinions, 
perspectives, or feelings related to the classroom, target children, or target practices. 

The coach engages in role-play OR provides a live verbal model of how to implement a practice. 


focused on the content, structure, and coaching processes 
that would be used to facilitate teachers’ use of embedded 
instruction teaching practices. The coach training was facil- 
itated by two university-based principal investigators and 
the lead coach. The coach training included an introduction 
to the teaching practices (4 hr) and how to use PBC to sup- 
port teacher’s use of the teaching practices (12 hr) includ- 
ing (a) how to implement the PBC framework following 
the project-developed protocol and fidelity checklist 
(coaching log), (b) coaching practices designed to facili- 
tate teacher participation in the collaborative partnership, 


and (c) supplemental materials to inform coaching. The 
coach training employed active learning strategies includ- 
ing (a) videos of preschool teachers implementing embed- 
ded instruction practices and coach—teacher dyads engaged 
in PBC, (b) guided practice using embedded instruction 
planning forms, and (c) role-play using PBC protocols with 
performance-based feedback. 


Ongoing implementation support for coaches. Throughout 
onsite coaching within the RCT, coaches received group 
and individual feedback and support. Coaches participated 
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in two meetings each week (~60-min each). One meeting 
was with their university-based coaching team (approxi- 
mately six people) and the other was with the cross-site 
coaching team (~12 people). Meetings included the coaches, 
investigators, and project coordinators of the RCT. During 
these meetings, content, structure, and process features of 
coaching were discussed. Coaches shared celebrations, 
problem-solved around implementation challenges, dis- 
cussed strategies and resources for supporting teachers to 
implement the embedded instruction teaching practices, 
and received clarifications and feedback about the average 
fidelity of implementation of the coaching protocol. 
Coaches also received individual fidelity feedback by email 
on a minimum of three occasions throughout the year. As 
needed, coaches received additional feedback and support 
to ensure their coaching protocol fidelity of implementation 
was at or above 80%. Fidelity feedback was provided by the 
project coordinator and principal investigators about the 
coach’s ability to (a) implement protocol indicators, (b) 
accurately record protocol indicators and coaching strate- 
gies implemented, and (c) write a follow-up email to the 
teacher summarizing the debrief meeting using the email 
protocol. Feedback included strengths, specific indicators 
to review, and recommendations for aligning coaching 
practices with the PBC framework. 


PD intervention: Embedded instruction teaching practices. 
Embedded instruction is an intentional and systematic 
instructional approach for promoting children’s acquisition, 
maintenance, and generalization of skills that support access 
to and participation in the general preschool curriculum 
(Snyder et al., 2018). Preschool teachers enrolled in the 
onsite PBC condition of the RCT attended four, 4-hr work- 
shops on embedded instruction teaching practices. During 
the workshops, teachers participated in knowledge- and 
application-focused activities with in situ support and feed- 
back from their coach. 


Onsite PBC. When implemented with fidelity, PBC has 
demonstrated efficacy in supporting preschool teachers to 
implement and sustain their use of select teaching practices 
(e.g., Hemmeter et al., 2016; Snyder et al., 2018). Snyder 
et al. (2015) define PBC as “a cyclical process for support- 
ing preschool teachers’ use of effective teaching practices 
that lead to positive outcomes for children” (p. 2). PBC is 
composed of three cyclical components: (a) shared goals 
and action planning, informed by a strengths and needs 
assessment; (b) focused observation; and (c) reflection and 
feedback. The PBC components occur within the context 
of a collaborative partnership and focus on a set of evi- 
dence-based teaching practices (Figure 1). 

In the RCT, the teaching practices were focused on 
embedded instruction and the PBC cycle included 15 onsite 
coaching sessions plus an orientation and closing meeting. 


Teaching 
Practices 


Figure |. Components of the practice-based coaching 
framework (Snyder et al., 2015). 


Each session was composed of a focused observation 
(~60 min), followed by a debrief meeting (~30—60 min) 
guided by a project-developed coaching protocol and an 
action plan. Action plans were informed by a strengths and 
needs assessment, written collaboratively by the coach and 
teacher, and updated across sessions. The action plan was 
composed of five parts: (a) the teachers’ goal for implement- 
ing embedded instruction teaching practices, (b) a criterion 
statement to determine when the goal had been met, (c) 
action steps for achieving the goal, (d) resources needed, and 
(e) a timeline. During the focused observation, the coach 
recorded video or took qualitative and quantitative notes 
about the teacher’s action plan goal and his or her implemen- 
tation of embedded instruction teaching practices. Following 
the focused observation, the coach and teacher met to 
debrief. The debrief meeting was a dyadic transactional 
exchange between the coach and teacher designed to facili- 
tate teacher reflection and problem-solving, provide sup- 
portive and constructive performance-based feedback, and 
to plan for future implementation guided by the action plan 
and the embedded instruction teaching practices. Following 
each debrief meeting, the coach sent an email aligned with 
the project-developed protocol to summarize feedback and 
next steps. 


Video selection. Coaches video-recorded all debrief meet- 
ings within the RCT. In Year | of the RCT, debrief meet- 
ings were 48 min on average (range: 13-118 min) and for 
the debrief meetings in the present study, 45 min on aver- 
age (range: 26-73 min). Coaching videos were blocked 
into three occasions representing early (1-5), middle (6-10), 
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and late (11-15) sessions to obtain a representative sample 
for the present study. One video was randomly selected 
from each occasion for each coach-teacher dyad, and no 
consecutive sessions were selected within each dyad. There 
were seven videos for each occasion, 21 videos total. 


Video coding. The CPOT-RVI was applied to video-recorded 
debrief meetings using Observer XT 12.5® behavioral cod- 
ing and analytic software. Coders viewed each video twice. 
During the first pass, the coder scored the occurrence of 
coach verbal behavior. During the second pass, the coder 
scored the duration of conversation focus, including whether 
the coach or teacher initiated the conversation focus. The 
data produced through the application of the CPOT-RVI 
codes were quantified to determine (a) the amount of time 
spent during the debriefing meeting in each conversation 
focus, (b) how often each participant initiated the conversa- 
tion focus, and (c) how often coaches used the verbal behav- 
iors during the debriefing meeting. Duration and verbal 
behavior data were subsequently integrated to determine the 
rate of verbal behavior and use of particular verbal behaviors 
during each conversation focus. Coach verbal behavior, 
duration of conversation foci, and initiations were also cal- 
culated at each sampling occasion to explore variability over 
time. Additional information about this measure can be 
found in the CPOT-RVI manual (Shannon & Snyder, 2016). 


Interobserver agreement (IOA). The primary coder was the 
first author and a coach within the RCT. The secondary 
coder was a master’s student in counselor education with 
preschool teaching experience. The secondary coder was 
trained to conduct point-by-point IOA (Yoder et al., 2018) 
for the CPOT-RVI. Agreement was defined as both coders 
selecting the same verbal behavior, duration, or initiation 
code using a 5-s agreement window. Agreement was calcu- 
lated as the ratio of agreements to the sum of agreements 
and disagreements multiplied by 100. Prior to coding for 
the study, the secondary coder coded five videos with a 
minimum of three of the five videos at or above 80% agree- 
ment overall and for each duration and event code. The sec- 
ondary coder coded 48% (n = 10) of the 21 videos. IOA was 
calculated for seven randomly selected videos, plus three 
additional videos that included the primary coder as the 
coach. Following the fourth IOA video, overall IOA for two 
consecutive videos was below 80% and the secondary coder 
received a booster session. The overall mean percentage for 
the 10 IOA sessions was 85 (range: 66-94). The mean IOA 
percentage scores averaged across the 10 IOA sessions 
ranged from 71 to 100 for duration codes, 55 to 100 for the 
initiation codes, and 62 to 81 for the event codes. Kappa 
was calculated as the ratio of expected agreements to the 
sum of possible agreements (Cohen, 1960). The average 
Cohen’s kappa across the 10 IOA sessions was 0.79 (range: 
0.56—0.93), suggesting moderate to strong agreement. 


Results 


Results are presented by duration of conversation focus cat- 
egory, coach versus teacher initiation, and coach verbal 
behavior. Within each section, the data are presented across 
the 21 video-recorded observations and by occasion (i.e., 
early, middle, late). Integration of duration and verbal 
behavior data are also provided to characterize the type of 
verbal behavior by conversation focus. Across the 21 
debrief meetings, 936 min of footage was coded. Twenty- 
five minutes or 2.6% of the total time was coded as 
Uncodeable (e.g., sound not audible, consented participants 
answered phone) and excluded from the analyses. 


Duration of Conversation Foci 


The CPOT-RVI includes six duration codes, plus Uncodeable 
(see Table 1). The percent of time in each conversation focus 
at each occasion (i.e., early, middle, late) and across occasions 
is shown in Table 3. The majority of the debrief meeting time 
was spent in Reflection and Feedback (42.7%), followed by 
Goal Setting and Action Planning (33.6%). From Occasion | 
to 3, the proportion of time spent in Personal/Pleasantries, 
Coaching Process, and Summarizing decreased and the pro- 
portion of time spent in Problem-Solving increased. 


Coach Versus Teacher Initiation 


The Jnitiation code is a binary forced response code fol- 
lowing the onset of one of the six duration codes. It was 
used to identify who within the dyad initiated the onset of 
a new conversation focus. Uncodeable did not receive a 
coach or teacher initiation code. The initiation of a new 
conversation focus was recorded on 365 occasions across 
the 21 coded sessions. The overall percentage of coach ver- 
sus teacher initiations was 77 and 23, respectively. From 
Occasion | to Occasion 3, the overall percentage of coach- 
initiated conversation foci decreased. The percent of coach 
and teacher initiations across occasions for each code is 
shown in Table 3. Coaches were more likely to initiate all 
conversation foci with the exception of Problem-Solving. 


Coach Verbal Behavior 


The CPOT-RVI includes eight verbal behavior event codes 
(see Table 2). Coach verbal behavior is presented as rate per 
30 min, consistent with the recommended duration for the 
debrief meeting in the RCT PBC protocol. Coaches’ average 
rate of verbal behavior per 30 min across all verbal behavior 
codes was 104.3 (SD = 23.5, range: 72.0—-133.0). The aver- 
age rate of coach verbal behavior per 30 min for each code is 
shown in Table 4. The rate of coach verbal behavior across 
the three occasions was analyzed to evaluate whether changes 
were evident. Across occasions, there was an increase in 
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Table 3. Percent of Time in Conversation Foci and Coach Versus Teacher Initiations Across Occasions. 


Occasion | 


Conversation focus M=5I min (SD = 14.4) 


Occasion 2 
M = 44 min (SD = 12.3) 


Overall 
M = 45 min (SD = 14.2) 


Occasion 3 
M = 42 min (SD = 16) 


Personal/Pleasantries 


Time 1.4 1.2 

Coach 81.8 90.9 

Teacher 18.2 9.1 
Coaching Process 

Time 11.2 10.9 

Coach 93.8 85.7 

Teacher 6.3 14.3 
Goal Setting and Action Planning 

Time 39.2 23.2 

Coach 79.2 88.2 

Teacher 20.8 11.8 
Problem-Solving 

Time 4.3 42 

Coach 75.0 33.3 

Teacher 25.0 66.7 
Reflection and Feedback 

Time 36.3 54.8 

Coach 74.4 61.9 

Teacher 25.6 38.1 
Summarizing 

Time 7.7 5.6 

Coach 100.0 100.0 

Teacher 0.0 0.0 


1.0 1.2 
62.5 80.0 
37.5 20.0 

8.8 10.4 
68.8 83.0 
31.3 17.0 
36.6 33.6 
76.5 80.0 
23.5 20.0 

9.7 5.9 
42.9 50.0 
57.1 50.0 
39.0 42.7 
62.3 65.9 
37.7 34.1 

48 6.2 

100.0 100.0 

0.0 0.0 


Note. Across the 2| coaching debrief meetings, 936 min of footage was coded. Twenty-five minutes or 2.6% of the total time was Uncodeable and 
excluded from the analyses. Each occasion included seven videos, one per dyad. 


Table 4. Rate of Coach Verbal Behavior Per 30 Mins Across Occasions. 


Coach verbal behavior Occasion | Occasion 2 Occasion 3 Overall 
Neutral Statement 55.0 66.4 64.1 61.2 
General Praise and Agreement 9.7 2:9 10.4 10.0 
Supportive Verbal Feedback 3.7 1.7 2.9 2.8 
Constructive Verbal Feedback 2.5 3.7 3:3 3.1 
Instructional Statements 8.3 7.9 12.5 9.5 
Clarifying Questions 15.3 11.0 13.6 13.5 
Probing Questions 5. 3.2 5.1 4.5 
Demonstration 1.9 1.2 2.9 2.0 


Note. Each occasion included seven videos, one per dyad. 


the rate of coach verbal behavior in General Praise and 
Agreement, Constructive Verbal Feedback, Instructional 
Statements, and Demonstration and a decrease in Supportive 
Verbal Feedback and Clarifying Questions. 


Rate of Coach Verbal Behavior by Conversation 
Focus 


The rate of each verbal behavior per 30 min within each 
conversation focus was calculated to capture not only the 


form of the coach verbal behavior but also its function 
within the coaching process. The results of these analyses 
are shown in Table 5 and in the parenthetical below. The 
rate of coach verbal behavior is an indication of the number 
of times coaches engaged in each of the eight verbal behav- 
iors. General Praise and Agreement (PA) and Supportive 
Verbal Feedback (SF) occurred within all conversation foci 
and most often during Reflection and Feedback (PA = 15.2 
and SF = 3.9) and Summarizing (PA = 11.8 and SF = 12.9). 
Similarly, Constructive Verbal Feedback occurred often 
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Table 5. Rate of Coach Verbal Behavior Within Each Conversation Focus Per 30 Mins. 


Conversation foci 


Coach verbal behavior PP CP GS PS RF SM 
Neutral Statement 160.9 88.7 54.4 54.4 59.3 51.4 
General Praise and Agreement 10.9 2.2 6.3 5.0 [5:2 11.8 
Supportive Verbal Feedback 2.7 0.6 0.7 0.6 3.9 12.9 
Constructive Verbal Feedback 0.0 0.3 1.2 2.8 5.6 2.7 
Instructional Statements 0.0 2.5 14.1 18.3 8.0 0.0 
Clarifying Questions 16.4 30.0 14.3 72 11.2 27 
Probing Questions 5.5 3:5 6.4 3.9 4.0 0.0 
Demonstration 0.0 0.1 0.3 0.6 0.4 0.2 


Note. PP = Personal/Pleasantries; CP = Coaching Process; GS = Goal Setting and Action Planning; PS = Problem-Solving; RF = Reflection and Feedback; 


SM = Summarizing. 


during Reflection and Feedback (5.6) and Summarizing 
(2.7) but also occurred frequently during Problem-Solving 
(2.8). Instructional Statements most often occurred within 
Problem-Solving (18.3) and Goal Setting and Action 
Planning (14.1). Clarifving Questions and Probing 
Questions occurred within all conversation foci, except 
Summarizing. Clarifying Questions occurred most often 
during the Coaching Process (30.0), while Probing 
Questions occurred most often within Goal Setting and 
Action Planning (6.4). Demonstrations occurred at the 
lowest rate overall and occurred most often during 
Problem-Solving (0.6). 


Discussion 


When implemented with fidelity, PBC has demonstrated 
efficacy in supporting preschool teachers to implement and 
sustain their use of select teaching practices (Hemmeter 
et al., 2016; Snyder et al., 2018). The present study explored 
how coaches facilitated conversations during the debrief 
meeting component of a PBC coaching partnership. The 
CPOT-RVI, a direct behavioral observation coding system 
aligned with essential components of the PBC framework, 
was used to investigate (a) the proportion of time allocated 
to different conversational foci, including who initiated the 
focus, and (b) the verbal behaviors used by coaches to sup- 
port teachers’ active participation in planning for, imple- 
menting, and evaluating the teaching practices that were the 
focus of coaching. Descriptive methods were used to deter- 
mine whether the conversation foci, coach and teacher ini- 
tiations, and verbal behaviors changed across the three 
occasions (i.e., early, middle, late) for seven coach-teacher 
dyads. 


Duration of Conversation Focus 


Findings related to the duration of conversation focus 
showed dyads engaged in all categories of conversation foci 


and spent the largest proportion of time in Reflection and 
Feedback (42.7%), followed by Goal Setting and Action 
Planning (33.6%). These conversation foci align with two 
components of the PBC framework that lead to noteworthy 
changes in teachers’ use of practices (e.g., Snyder et al., 
2015). Dyads also spent a small proportion of time in 
Personal/Pleasantries (1.2%) at each occasion suggesting 
that most dyads continued to build rapport and briefly share 
nonclassroom-related interests as part of their collaborative 
partnership. Results from the present study provide further 
evidence that coaches can implement the PBC framework 
as intended when provided with a structured protocol, ini- 
tial training plus ongoing support, and performance-based 
feedback (Durlak & DuPre, 2008; Snyder et al., 2015). 

On average, distribution of debrief meeting time in the 
present study was well aligned with the coaching protocol 
indicators. A consistent coaching protocol was used for all 
sampled occasions, yet across occasions and dyads, vari- 
ability in the proportion of time spent in each conversation 
focus was observed. These data suggest that coaches can 
have high fidelity to the protocol (1.e., content and structural 
dimensions) while still individualizing the process dimen- 
sion by dedicating more or less time to a particular conver- 
sation foci in response to the teacher’s implementation of 
practices during the focused observation, the current action 
plan goal, or context-based needs (e.g., child behavior, new 
adults in classroom) and motivations (e.g., supports or chal- 
lenges from administrators and staff), which often change 
across time. 


Coach and Teacher Initiation 


There was an increase in teacher initiations for most conver- 
sation foci from Occasion | (i.e., early) to Occasion 3 (i.e., 
late). As teachers increased their initiations, coaches accom- 
modated by following the teachers’ lead and continuing con- 
versation foci initiated by the teacher. The largest percent of 
teacher-initiated conversations focused on Problem-Solving 
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followed by Reflection and Feedback. These conversation 
foci might have been initiated more often by teachers 
because they provided access to coach support to address an 
immediate need within the teacher’s classroom. In prior 
studies of teachers’ perspectives about PBC, preschool 
teachers reported that they rarely had access to ongoing per- 
formance-based feedback and support to address dilemmas 
in their classrooms, in particular from someone who was 
knowledgeable about early childhood classrooms (Shannon 
et al., 2015). 

Mutual adaptation in interactions between the coach and 
teacher was also seen in the Coaching Process conversation 
focus across the three occasions. The duration of time spent 
in the Coaching Process decreased across occasions. Dyads 
dedicated less time to roles and responsibilities as the col- 
laborative partnership developed. However, when Coaching 
Process conversations did occur, they were increasingly ini- 
tiated by the teacher from Occasion | (6.3%) to Occasion 3 
(31.3%). Developments within the Coaching Process con- 
versation foci suggest teachers became more proficient in 
their ability to identify next steps for implementation and to 
articulate how they wanted the coach to provide support 
within the partnership over time. 


Coach Verbal Behavior 


Coaches consistently used the PBC protocol verbal behav- 
iors (i.e., Supportive Verbal Feedback, Constructive Verbal 
Feedback, Clarifying Questions) across all three occasions 
observed and coded for the present study. Providing sup- 
portive performance-based feedback about the teacher’s 
demonstration of actions or behaviors associated with 
overall classroom quality, and the use of targeted teaching 
practices is an essential component of PBC. Within the 
CPOT-RVI coding system, positive feedback was captured 
through the use of two verbal behavior codes: (a) Supportive 
Verbal Feedback and (b) General Praise and Agreement. 
Overall, Supportive Verbal Feedback occurred, on average, 
at a rate of 2.8 times per 30 min, while the rate of General 
Praise and Agreement occurred, on average, 10 times per 
30 min. Supportive Verbal Feedback occurred, on average, 
at the highest rate during the Summarizing conversation 
focus, while General Praise and Agreement occurred, on 
average, at the highest rate during Reflection and Feedback. 

Of note, Supportive Verbal Feedback is more likely than 
General Praise and Agreement to support teachers to 
acquire knowledge and use of teaching practices (Casey & 
McWilliam, 2011). Supportive Verbal Feedback describes 
specific teaching practices observed by the coach and, in 
many cases, why maintaining and generalizing those prac- 
tices would support positive child outcomes (e.g., By tap- 
ping the shelf along with your verbal direction “Put the car 
on the shelf,” you helped Ziya to understand and demon- 
strate her knowledge of the positional word “on.”). In 


contrast, General Praise and Agreement is a general 
acknowledgment of teacher actions observed by the coach 
(e.g., Great job with Ziya today!), but it does not support the 
teacher to know the attributes of his or her actions that make 
them positive or generalizable to future application in the 
classroom setting. 

Both Supportive Verbal Feedback and General Praise 
and Agreement were addressed in the PBC coach training 
and through ongoing fidelity feedback within the RCT. 
Nevertheless, implementation fidelity measured by the 
fidelity checklist (coaching log) was a binary code record- 
ing adherence, that is, whether Supportive Verbal Feedback 
occurred or did not occur. The fidelity coding system in the 
RCT did not account for the rate of this verbal behavior 
within each debrief meeting. Coaches may need additional 
training and support to distinguish between forms of posi- 
tive feedback. They might also benefit from training to 
increase the frequency of Supportive Verbal Feedback by 
not only developing and delivering planned Supportive 
Verbal Feedback in the form of Summarizing statements but 
also increasing their capacity to respond to teacher’s reflec- 
tions about the classroom observation. 

Constructive Verbal Feedback serves a complimentary 
role to Supportive Verbal Feedback within the PBC frame- 
work. It increases the teachers’ knowledge of teaching 
practices, supporting them to self-assess their current use 
of the practices and to identify ways to enhance current 
implementation or acquisition and mastery of new teaching 
practices (Snyder et al., 2015). Several studies have dem- 
onstrated the importance of providing both supportive 
and constructive feedback concurrently when helping 
teachers acquire and master the use of new teaching prac- 
tices (e.g., Oborn & Johnson, 2015; Snyder et al., 2018). 
In the absence of Constructive Verbal Feedback, teachers 
may not be aware that they are not implementing a teach- 
ing practice with fidelity (Shannon et al., 2015). They are 
also apt to abandon a practice because they do not see 
immediate positive outcomes as a result of their imple- 
mentation or do not have access to implementation sup- 
port (Kretlow & Bartholomew, 2010). 

Constructive Verbal Feedback occurred, on average, 3.1 
times per 30 min, complementing the Supportive Verbal 
Feedback, which occurred, on average, 2.8 times per 30 
min. The highest rates of Constructive Verbal Feedback 
occurred during transactional conversation foci including 
Reflection and Feedback (5.6) and Problem-Solving (2.8). 
One strength of the relationship between the provision of 
constructive feedback and these conversation foci is that 
adult learners benefit most from feedback when they can 
respond through dialogue and have ample opportunity to 
express thoughts, concerns, or questions (NASEM, 2018). 
Constructive Verbal Feedback was also delivered (2.7 times 
per 30 min) in the context of Summarizing, a coach-talk-only 
conversation focus. What is unknown from these data, and 
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worthy of further exploration, is whether Constructive 
Verbal Feedback delivered in the context of Summarizing 
statements was followed by a transactional conversation 
focus. 

Two types of questioning verbal behavior were recorded, 
Clarifying Questions and Probing Questions. Both types of 
questions are appropriate within the debrief meeting. On 
average, Probing Questions occurred 4.5 times per 30 min, 
while Clarifying Questions occurred 13.5 times per 30 min. 
Both types of questions occurred across most conversation 
foci, signifying the coaches’ efforts to engage the teachers 
by eliciting their perspectives throughout the debrief meet- 
ing. Although both Probing Questions and Clarifying 
Questions can elicit teacher participation, the types of ques- 
tions posed might have an important role in prompting 
teachers to engage in self-reflection and evaluation. Moyers 
and colleagues (2007) found open-ended or probing ques- 
tioning focused on change (e.g., What would have made 
block play more successful for embedded learning opportu- 
nities for Kisharra?) increased the probability of partici- 
pants’ openness to changing their behavior. Probing 
Questions occurred at the highest rate during Goal Setting 
and Action Planning (6.4 per 30 min), a logical and strate- 
gic time for eliciting the teachers’ perspective about future 
goals. The use of Probing Questions during this conversa- 
tion focus ensured coaches were eliciting the teachers’ per- 
spective about their implementation of teaching practices. 
Teacher talk in this conversational focus and others includ- 
ing Reflection and Feedback (4.0 per 30 min) and Problem- 
Solving (3.9 per 30 min) also allowed the coach to address 
teachers’ individual needs, misunderstandings, or imple- 
mentation challenges. 


Implications 


Our findings contribute to understanding interactions that 
occur during PBC debrief meetings. These data help address 
the identified need for additional information about coach- 
ing processes situated in the context of coaching content and 
structures. Our data also support the development of coach- 
ing implementation supports as a part of coach-based PD. 
First, operationally defining the key components of a coach- 
ing framework and providing coaches with PD, ongoing 
fidelity feedback, and coaching implementation supports is 
likely to be associated with fidelity of implementation of the 
coaching protocol and, in turn, teachers’ fidelity of imple- 
mentation of the teaching practices. These findings are con- 
sistent with a broader effort to advance the science of PD in 
early childhood, including coaching, by clearly defining the 
content, structure, and processes used as an independent 
variable in research (Powell & Diamond, 2013; Snyder 
et al., 2011). This also applies in practice, as states and pro- 
grams seek to adopt interventions that are scalable, effi- 
cient, and cost-effective for enhancing the quality of early 


childhood experiences and child outcomes (Artman-Meeker 
et al., 2015; Zaslow et al., 2010). PBC is an evidence-based 
framework for implementing coaching. It employs key ten- 
ants from the science of how people learn (NASEM, 2018) 
and organizational behavior management (Daniels & Bailey, 
2014), has operationally defined components, and has dem- 
onstrated efficacy in achieving teacher and child outcomes 
across a variety of teaching practices and contexts (Snyder 
et al., 2015). The extent to which coaching has measurable 
and replicable effects on teachers’ use of evidence-based 
teaching practices is contingent on coaches implementing an 
evidence-based coaching framework, such as PBC, with 
fidelity. 

Second, as coaching continues to be provided to support 
implementation of evidence-based practices for teachers 
with varied educational and experiential backgrounds, it 
will be important to use evidence-based coaching frame- 
works that provide coaches with sufficient structure, 
yet allow necessary variability to meet teachers’ knowledge, 
preferences, motivations, and individual goals. In the pres- 
ent study, controlled doses of coaching were provided, 
although coaching processes varied across teachers and over 
time. Coach-based PD supports used to aid teachers’ acqui- 
sition or enhancement of practices should be documented to 
increase the capacity to replicate findings and develop 
acceptable thresholds of process adaptations (e.g., more or 
less constructive feedback, duration of debrief). Systematic 
documentation of coach processes, as well as the environ- 
mental factors which influence their success (e.g., leader- 
ship, policies), is needed to scale up coach-based PD 
initiatives (Artman-Meeker et al., 2015; Snyder et al., 2012). 

Third, the present study contributes a continuous, timed- 
event, observational coding system that can be used to 
explore the observed duration of conversation focus, coach— 
teacher initiations, rate of coach verbal behavior, and 
change in these variables across time. Previous research of 
coaching interactions employing observational coding sys- 
tems used interval systems (Campbell & Coletti, 2013; 
Jayaraman et al., 2015; Salisbury et al., 2012). The use of 
interval coding has the potential to overrepresent or under- 
represent the occurrence of behaviors, whereas continuous 
timed-event coding is a preferred behavior sampling method 
(Yoder et al., 2018). In addition, previous studies coded 
only coach behaviors, not the conversation focus during 
which the verbal behavior was used. By including informa- 
tion about conversation focus, it is possible to make infer- 
ences about the function of the coaches’ verbal behavior. 
For example, data in the present study show that not only 
did Probing Questions occur, but information was gathered 
about whether coaches used these questions to engage 
teachers in reflection about their current implementation of 
teaching practices or to guide the development of future 
action plans. In addition, the documentation of coach and 
teacher initiations is a small, but important first step, in 
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examining how roles within the PBC collaborative partner- 
ship evolve. 

Finally, the present study differs from prior coaching 
studies employing observational coding systems in that it 
delimited the factors influencing the coaching partnership. 
All coaches and teachers focused on a common set of 
embedded instruction teaching practices and all teachers 
worked in school-based preschool classrooms. The 15-week 
duration of the coaching partnership was uniform, and 
occasions sampled represented early, mid, and late debrief 
meetings to explore change over time, as opposed to meet- 
ings being coach selected and representing a single occa- 
sion (Campbell & Coletti, 2013; Jayaraman et al., 2015). 


Limitations and Recommendations for Future 
Research 


Several limitations of this study should be noted and receive 
attention in future research. First, an extension of the pres- 
ent study might include a larger and more diverse sample. 
Within the present study, one teacher was paired with each 
coach. Given the transactional nature of the collaborative 
PBC coaching partnership, it would be beneficial to deter- 
mine if coaches engage in similar conversation foci and ver- 
bal behaviors across teachers or if characteristics of the 
teacher (e.g., novice versus veteran) appear to influence the 
coach’s behavior. In the present study, coaches were univer- 
sity based. Although the PBC framework has been scaled 
and implemented outside of applied research studies, data 
should be collected for indigenous coaches to explore 
whether similar patterns of conversation focus, interactional 
leads, and coach verbal behavior are present. Learning 
about the extent to which indigenous coaches maintain 
fidelity to the protocol, engage in similar conversation foci, 
use similar behaviors when provided with similar supports, 
and find the PBC framework acceptable, feasible, and use- 
ful would be important to programs and states considering 
the installation of PBC to support early childhood practitio- 
ners’ use of evidence-based practices. In addition, it will be 
important to document how the knowledge and skills of 
leadership teams and organizational policies influence the 
coaches’ ability to achieve desired teacher and child out- 
comes (Durlak & DuPre, 2008). 

Second, the CPOT-RVI was developed, piloted, and 
applied in the present study. The use of the coding system has 
not been replicated across multiple evidence-based teaching 
practices or coders. Third, the CPOT-RVI IOA data in the 
present study suggest the need to refine further the codes, 
particularly those focused on distinguishing between imple- 
mentation adherence and quality or instructional utility to the 
teacher. Areas of concern for fidelity were (a) detecting 
teacher initiations, (b) distinguishing between Coaching 
Process and Goal Setting and Action Planning, and (c) dis- 
criminating between similar codes with small but important 


qualitative differences such as Constructive Verbal Feedback 
and Instructional Statements. For example, Constructive 
Verbal Feedback, as operationally defined in the CPOT-RVI, 
is hypothesized to have greater utility for the teacher because 
it connects the coach’s recommendations for enhanced prac- 
tice implementation to an observed teacher action, such that 
in the future under similar conditions, the teacher will know 
when and how to use a practice effectively. Instructional 
statements also provide recommendations for enhancement 
but may have less utility for the teacher because they are not 
explicitly linked to an observed action making it less clear for 
the teacher when and how to apply recommendations. These 
nuanced differences require clarification in the CPOT-RVI 
manual and training for coders. Fourth, the CPOT-RVI 
focuses primarily on coach verbal behaviors rather than ver- 
bal behaviors of the teacher, an important consideration 
within a transactional collaborative partnership. 


Conclusion 


The CPOT-RVI is a direct behavioral observation coding 
system aligned with the evidence-based coaching practices 
that are essential to implementing PBC with fidelity. 
Findings from the present study provide important informa- 
tion about the extent to which a structured coaching proto- 
col, like the one used within the RCT, supports coaches to 
implement an evidence-based coaching framework with 
fidelity. Furthermore, it contributes to ongoing efforts to 
understand how structured protocols can be implemented 
with fidelity while tolerating variability designed to meet 
the unique needs of individual teachers. Coaching has the 
potential to play a powerful role in supporting teachers to 
use evidence-based teaching practices and, in turn, improve 
child outcomes. To consistently achieve these more distal 
impacts, the field must begin by better defining the active 
ingredients of coaching designed to catalyze change. 
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